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RECEIVE IMAGES OVER NETWORK 






DETERMINE RELATIVE POSITIONS 




r ^204 


CORRECT PERSPECTIVE DISTORTION 






COMPUTE FOCAL LENGTH 
AND ANGLE FOR EACH IMAGE 




, ^-208 


MAP IMAGES TO CYLINDER 
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BLEND 1 


MAGES 
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CREATE EMPTY "POSITIONED LIST" STRUCTURE FOR 
STORING IMAGES WHOSE RELATIVE POSITION IS KNOWN 



702 



NO 



CHECK IF 
THERE IS AN UNPOSITIONED 
J MAGE?, 

YES 

CHECK IF 
.POSITIONED LIST IS EMPTY- 

9 



704 



■r- (FROM 774 (FIG 7B)) 
^708 



706 



NO 



YES 



710 



F ADD 
UNPOSTIONED 
IMAGE TO 
POSITIONED 
LIST 



CREATE EMPTY "OVERLAP LIST' FOR STORING IMAGES IN 
POSITIONED LIST THAT OVERLAP THE UNPOSITIONED IMAGE 



SET: BEST_CONFIDENCE = 0 
BESTjCONFIDENCEJMAGE = NO_MATCH 
CURRENT IMAGE = FIRST IMAGE IN POSITIONED LIST 



714 



712 



DETERMINE OFFSET OF UNPOSITIONED 
IMAGE RELATIVE TO CURRENT IMAGE AND 
A CONFIDENCE VALUE FOR THE OFFSET 



CHECK 

IF CONFIDENCE VALUE > A THRESHOLD. 
VALUE^ 

"YES 



716 



726 



SET CURRENT 

IMAGE TO 
NEXT IMAGE IN 
POSITIONED 
LIST 



NO 
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ADD CURREM?IMAGE, THE OFFSET, AND THE 
CONFIDENCE VALUE TO THE OVERLAP LIST 



CHECK 

IF CONFIDENCE VALUE > BEST 
^CONFIDENCE? 

YES 



720 



NO 
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SET: BEST_CONFIDENCE = CONFIDENCE VALUE 
BEST_CONFIDENCE IMAGE = CURRENT IMAGE 



CHECK 

IF CURRENT IMAGE IS LAST IN 
POSITIONED LIST 7_ 

YES 
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SET REFERENCE IMAGE TO 
FIRST IMAGE IN OVERLAP LIST 
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(FROM 728 (FIG 7A)) 




SET CURRENT IMAGE = NEXT IMAGE AFTER 
REFERENCE IMAGE IN OVERLAP LIST 



YES 
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CHECK IF 
ADJACENT LISTS INDICATE THAT 
CURRENT IMAGE IS CONNECTED TO 
REFERENCE IMAGE 



NO 



754 



TRANSLATE CURRENT IMAGE AND ALL IMAGES 
CONNECTED TO IT RELATIVE TO THE REFERENCE 
IMAGE BASED ON THE OFFSETS OF THE REFERENCE 
AND CURRENT IMAGE TO THE UNPOSITIONED IMAGE 



I 



APPEND THE UNPOSITIONED IMAGE AND THE 
-VE OF THE OFFSET OF THE CURRENT IMAGE 
RELATIVE TO THE UNPOSITIONED IMAGE TO 
THE ADJACENT LIST OF THE CURRENT IMAGE 



760 



SET 
CURRENT = 
NEXT IMAGE 
AFTER 
CURRENT 
IMAGE IN 
OVERLAP 
LIST 




APPEND THE UNPOSITIONED IMAGE AND THE -VE 
OF THE OFFSET OF THE REFERENCE IMAGE 

RELATIVE TO THE UNPOSITIONED IMAGE TO THE 
ADJACENT LIST OF THE REFERENCE IMAGE 
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SET 
REFERENCE = 
NEXT IMAGE 

AFTER 
REFERENCE 
IMAGE IN 
OVERLAP 
LIST 
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TRANSLATE UNPOSITIONED IMAGE RELATIVE BESTCONFIDENCEJMAGE 
BASED ON THE OFFSET OF THE BESTXONFIDENCE IMAGE 



APPEND IMAGES IN OVERLAP LIST AND THEIR OFFSETS 
TO THE ADJACENT LIST OF THE UNPOSITIONED IMAGE 
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ADD UNPOSITIONED IMAGE TO THE POSITIONED LIST |\. J74 
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DETERMINE CENTERMOST IMAGE 
BASED ON RELATIVE POSITIONS 



802 



CREATE LIST OF CORRECTED IMAGES THAT INCLUDES 
ONLY THE CENTERMOST IMAGE 



CREATE LIST OF UNCORRECTED IMAGES THAT INCLUDES 
EVERY IMAGE EXCEPTTHE CENTERMOST IMAGE 
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INITIALIZE BY SETTING: MAX_OVERLAP_AREA = 0 
SELECTED_WARPED_IMAGE = UNDEFINED 
SELECTED UNWARPED IMAGE = UNDEFINED 
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SET: 

CURRENT.WARPED = FIRST IN IMAGE IN CORRECTED LIST 
-CURRENT UNWARPED = FIRST IMAGE-IN UNCORRECTED LIST 
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COMPUTE OVERLAP AREA BETWEEN 
CURRENT WARPED AND CURRENT UNWARPED 
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NO 



814 

CHECK IF 

.OVERLAP AREA > MAX_OVERLAP_AREA_ 

? 

YES ^816 



CHANGE SELECTED IMAGES BY SETTING: 
MAX_OVERLAP_AREA = OVERLAP AREA 
SELECTED_WARPED_IMAGE = CURRENT.WARPED 
SELECTED UNWARPED IMAGE = CURRENT UNWARPED 



820 




SET 
CURRENT. 
WARPED = 
NEXT IMAGE 
IN LIST OF 
CORRECTED 
IMAGES 



CORRECT THE PERSPECTIVE OF 
SELECTEDJJNWARPED RELATIVE TO SELECTED_WARPED 
BASED ON THEIR RELATIVE POSITIONS 



830 



824 



SET: 
CURRENT. 
UNWARPED = 
NEXT IMAGE 
IN LIST OF 
UNCORRECTED 
IMAGES 
CURRENT. 
WARPED = 
FIRST IMAGE 

IN LIST OF 
CORRECTED 
IMAGES 



MOVE CURRENTJJNWARPED IMAGE FROM LIST OF 
UNCORRECTED IMAGES TO LIST OF CORRECTED IMAGES 



DONE ) 

FIG...8 




FIG. 9 



15/20 
Original Image 

2-D coordinates 



4-D coordinates 



Vertex 0 
Vertex 1 
Vertex 2 
Vertex 3 
The i th vertex 



( x o' y 0 ) 

(x 2 , ^ 
(x 3 , y 3 ) 
(Xj, Yj) 

^130 

FIG- 10A 



(x 0 ,y 0 , o,i) 

(x 1 ,y 1 ,o;i) 

(x 2 , y 2 , 0,1) 
(Xg.yg.O.T) 
(x^ yj, 0,1) 

^-132 



>134 



TRANSLATE AWAY BY FOCAL LENGTH 
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ROTATE BY 9 V 
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TRANSLATE TOWARDS BY FOCAL LENGTH 
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DISTORT FOR PERSPECTIVE 



FIG..10B 




JAN 1 3 2003 

\ - 

1 . Translate outwards 
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Perspective Correction Transformation 
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2. Three rotations: 
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3. Translate inwards: 



10 0 0 
0 10 0 
0 0 10 
0 0-/1 
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4. Effect of focal length on Perspective: 



P = 



10 0 0 

0 10 0 

0 0 0 V/ 

0 0 0 1 



146 



1=igIioc 



J 




17/20 
Perspective Correction 



pective Corrected Image Vertices given by: 
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But: 



Wj- = — — (-sin^sine^ + cose z sine^cosep 

V, • . . . 

+ — (cose z sine x + sine^ine^cose^) 



+ cose cose Y 



752- 



and x-' and y- from the perspective corrected image are given by: 



x- , 

x/ = y~ and y/ 
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Therefore we can write: 
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Taking: 



' = % % n 
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We can write: 



-F(t) = 
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FAG _ fOD 
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^> ^ Newton s Method 

By Newton's method of numerical computation, t is an estimate 
of the values 

10* % e 2 /] 



then: 

r 166 

is a better estimate of the values. 



Where J' 1 is trie matrix of partial derivatives: 



dtj 

v — J 
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INITIALIZE FOCAL LENGTH 
AND ROTATION ANGLES 



1000 



COMPUTE COORDINATES OF VERTICES OF 
PERSPECTIVE DISTORTED IMAGE BASED 
ON FOCAL LENGTH AND ROTATION ANGLES 



1002 



COMPUTE ERROR BETWEEN COMPUTE 
COORDINATES OF VERTICES 
AND ACTUAL COORDINATES OF 
VERTICES FROM MANN METHOD 



1004 



DETERMINE JACOBIAN MATRIX 



7006 



MULTIPLY ERRORS WITH INVERSE 
OF JACOBIAN MATRIX TO DETERMINE 
CORRECTION TO ROTATION 
ANGLES AND FOCAL LENGTH 
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UPDATE FOCAL LEN 
ANGLES BASED ( 


GTH AND ROTATION 
DN CORRECTION 
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CHECK IF - — YES 
CORRECTION > THRESHOLD^ 
.VALUER 
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SET PIXELS OF ALL IMAGES VISIBLE EVERYWHERE 
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SET CURRENT IMAGE TO FIRST IMAGE 



1208 



SET CURRENT IMAGE TO NEXT AFTER CURRENT 
IMAGE AND REFERENCE IMAGE TO FIRST IMAGE 



■1210 



DETERMINE OUTLINE OF PANORAMIC IMAGE FORMED BY 

ALUGNING THE CURRENT AND REFERENCE IMAGES 
BASED ON THE RELATIVE POSITION OF THE SEGMENT 
CORRESPONDING TO THE CURRENT IMAGE 



V 



1212 



DETERMINE LINE DIVIDING PANORAMIC OUTLINE (I.E., 
LONGEST LINE JOINING POINTS WHERE BORDERS OF 
CURRENT IMAGE AND REFERENCE IMAGE INTERSECT) 



s 



1214 



I 



DIVIDE PANORAMIC OUTLINE INTO 
TWO SECTIONS BASED ON LINE 



I 



1216 



1218 



DETERMINE WHICH OF THE TWO SECTIONS HAS 
LESS OF THE CURRENT IMAGE THAT IS NOT 
OVERLAPPED BY THE REFERENCE OUTLINE 



r 
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SET CURRENT IMAGE TO NOT BE 
INVISIBLE IN THE DETERMINED SECTION 



SET 
REFERENCE 
TO NEXT 
IMAGE 
AFTER 
REFERENCE 
IMAGE 
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_ 1222 

ANY MORE 
IMAGES AFTER REFERENCE IMAGE 
BUT BEFORE CURRENT 
IMAGE ? 



YES 



DETERMINE MASKING VALUE FOR EACH PIXEL 
OF CURRENT IMAGE BASED ON VISIBLE AREA 
OF IMAGE AND PROXIMITY TO DIVIDING LINES 




OVERLAP IMAGES BASED ON MASKING 
VALUES TO CREATE A PANORAMIC IMAGE 



